Skip to content

feat: aot metered cost #2168

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 107 commits into from
Oct 30, 2025
Merged

feat: aot metered cost #2168

merged 107 commits into from
Oct 30, 2025

Conversation

@Maillew
Copy link
Contributor

@Maillew Maillew commented Oct 15, 2025 •
edited
Loading

Description

Adds support for AOT Metered Cost Execution

Testing

Modified existing fn check_aot_equivalence function in stark_utils.rs to assert consistency between AOT and Interpreted metered cost execution

Added additional tests in extensions/native/circuit/tests/integration_test.rs to test various scenarios surrounding metered_cost

Closes INT-5255

@github-actions

This comment was marked as resolved.

Sign in to view
@Maillew Maillew changed the title Feat/aot metered cost feat: aot metered cost Oct 15, 2025
@Maillew Maillew force-pushed the feat/aot-metered-cost branch from 9b405e3 to 6e34a08 Compare October 16, 2025 18:06
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@Maillew Maillew force-pushed the feat/aot-metered-cost branch from 2c7426e to 5d52991 Compare October 16, 2025 21:33
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@Maillew Maillew force-pushed the feat/aot-metered-cost branch from d45154e to 6a06185 Compare October 16, 2025 23:04
@Maillew Maillew marked this pull request as ready for review October 16, 2025 23:44
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@Maillew Maillew force-pushed the feat/aot-metered-cost branch from 5235ff8 to b125b27 Compare October 17, 2025 20:44
@Maillew Maillew marked this pull request as draft October 17, 2025 22:43
@Maillew Maillew self-assigned this Oct 17, 2025
@shuklaayush
Copy link
Contributor

shuklaayush commented Oct 21, 2025

looks like test_invalid_setup is stuck in an infinite loop somewhere

jonathanpwang
jonathanpwang reviewed Oct 25, 2025
View reviewed changes
crates/asm/asm_bridge_utils/Cargo.toml
Copy link
Contributor

@jonathanpwang jonathanpwang Oct 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Thanks for re-organizing to share code.
  2. I didn't catch this before, but if each of asm_bridge_* is actually a new crate, they should not go inside arch/ or even inside the vm/ folder. I'd suggest making crates/asm and then separate folders for each crate inside.

jonathanpwang
jonathanpwang reviewed Oct 25, 2025
View reviewed changes
extensions/native/circuit/tests/integration_test.rs Outdated

// Helper to run AOT metered-cost and compare against interpreter baseline.
#[cfg(feature = "aot")]
macro_rules! run_aot_metered_cost_and_compare {
Copy link
Contributor

@jonathanpwang jonathanpwang Oct 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did this need to be a macro and not just a function?

jonathanpwang
jonathanpwang approved these changes Oct 25, 2025
View reviewed changes
Copy link
Contributor

@jonathanpwang jonathanpwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM and generally helped reorganize things structurally, but please address the comment that we shouldn't have crates inside vm/src/arch. These crates should be outside, e.g. crates/asm

@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions

This comment has been minimized.

Sign in to view
@github-actions
Copy link

github-actions bot commented Oct 29, 2025

group app.proof_time_ms app.cycles app.cells_used leaf.proof_time_ms leaf.cycles leaf.cells_used
verify_fibair 244 322,610 2,058,654 - - -
fibonacci 987 1,500,209 2,100,402 - - -
regex 2,292 4,137,502 17,695,216 - - -
ecrecover 692 122,859 2,263,820 - - -
pairing 1,423 1,745,742 25,468,210 - - -

Commit: 88db90d

Benchmark Workflow

@codspeed-hq
Copy link

codspeed-hq bot commented Oct 29, 2025 •
edited
Loading

CodSpeed Performance Report

Merging #2168 will degrade performances by 89.25%

Comparing feat/aot-metered-cost (88db90d) with main (c2e376e)1

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

❌ 18 regressions
⏩ 42 skipped2

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Mode Benchmark BASE HEAD Change
❌ WallTime benchmark_execute[bubblesort] 27.4 ms 246.3 ms -88.86%
❌ WallTime benchmark_execute[fibonacci_iterative] 31.4 ms 291.8 ms -89.25%
❌ WallTime benchmark_execute[fibonacci_recursive] 45.1 ms 396.3 ms -88.63%
❌ WallTime benchmark_execute[keccak256] 33 ms 264.5 ms -87.53%
❌ WallTime benchmark_execute[pairing] 137.8 ms 229 ms -39.82%
❌ WallTime benchmark_execute[quicksort] 32 ms 281.6 ms -88.64%
❌ WallTime benchmark_execute[revm_snailtracer] 16.1 ms 20.4 ms -21.01%
❌ WallTime benchmark_execute[revm_transfer] 43.1 ms 176.1 ms -75.55%
❌ WallTime benchmark_execute[sha256] 31.1 ms 262.9 ms -88.18%
❌ WallTime benchmark_execute_metered[bubblesort] 53 ms 280.2 ms -81.1%
❌ WallTime benchmark_execute_metered[fibonacci_iterative] 72.5 ms 324.5 ms -77.66%
❌ WallTime benchmark_execute_metered[fibonacci_recursive] 101.4 ms 465.6 ms -78.22%
❌ WallTime benchmark_execute_metered[keccak256] 68.9 ms 309.5 ms -77.74%
❌ WallTime benchmark_execute_metered[pairing] 156.1 ms 249.3 ms -37.41%
❌ WallTime benchmark_execute_metered[quicksort] 60.3 ms 321.7 ms -81.25%
❌ WallTime benchmark_execute_metered[revm_snailtracer] 17.1 ms 21.5 ms -20.57%
❌ WallTime benchmark_execute_metered[revm_transfer] 64.1 ms 201.2 ms -68.12%
❌ WallTime benchmark_execute_metered[sha256] 67.1 ms 307.2 ms -78.15%

Footnotes

  1. No successful run was found on feat/aot (f804542) during the generation of this report, so main (c2e376e) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

  2. 42 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@jonathanpwang
Copy link
Contributor

jonathanpwang commented Oct 30, 2025

We should either merge this now, or close it.

@Maillew
Copy link
Contributor Author

Maillew commented Oct 30, 2025

We should either merge this now, or close it.

Will merge after CI/CD finishes. Sry, didnt merge earlier cuz there were problems on main aot branch with CI/CD

@Maillew Maillew merged commit 154f154 into feat/aot Oct 30, 2025
53 of 57 checks passed
@Maillew Maillew deleted the feat/aot-metered-cost branch October 30, 2025 23:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jonathanpwang jonathanpwang jonathanpwang approved these changes

Assignees

@Maillew Maillew

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Maillew @shuklaayush @jonathanpwang @GunaDD